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(57) Abstract: Down-sampling of an image may be performed in the 
DCT domain. Transform matrices are obtained for down -sampling a 
DCT image of size M x N to a down-sampled DCT image of size I x J. 
The transform matrices may be used to down-sample the DCT image 
directly in the DCT domain. A spatial domain down-sampling method 
is selected and applied to the DCT image to produce a down-sampled 
DCT reference image. The transform matrices are selected by solving 
an optimization problem, leading to transform matrices which achieve 
a desired trade-off between the visual quality of images obtained using 
the transform matrices and the computational complexity associated 
with using the transform matrices. The visual quality is a measure of 
the difference between the down-sampled DCT image obtained using 
the transform matrices and the visual quality of the DCT reference im- 
age obtained using a spatial domain down-sampling method. 
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METHOD AND DEVICE FOR SELECTING OPTIMAL TRANSFORM 
MATRICES FOR DOWN-SAMPLING DCT IMAGE 

FIELD OF THE INVENTION 

[0001] The present invention relates to image manipulation and more 
particularly to the down-sampling of an image to obtain an image havFng a 
smaller size. 

BACKGROUND OF THE INVENTION 

[0002] As wireless and wired network connectivity is rapidly expanding 
and the number of networl< users is steadily Increasing, there has been great 
momentum in the multimedia industry for supporting content display in a 
wide variety of network connected devices. This is complicated, however, due 
to the great diversity of devices with various display resolutions. The range of 
devices includes both standard and high-definition computer monitors with 
large displays, and smart phone devices with smaller displays. Accordingly, it 
is often necessary to down-sample an image to a size which may be displayed 
on a given device. Down-sampling may also be desirable in order to minimize 
the bandwidth or time required to transfer an image over a network. This is 
particularly a concern where the image is being transferred over a wireless 
medium. 

[0003] Many image and video types represent Images or video in a 
Discrete Cosine Transform CDCT") domain. For example the JPEG, MPEG, DV 
and MJPEG Image and video types represent images or videos in the DCT 
domain. Where the image or video is represented in a DCT domain, down- 
sampling is particularly difficult. 

[0004] Typically, down-sampling a DCT image or video Involves first 
converting DCT data to a spatial domain using an inverse DCT operation and 
applying a standard down-sampling method in the spatial domain. After the 
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image is down-sampled in the spatial domain It is typically converted back to 
the DCT domain using a DCT operation. Due to the use of the DCT and the 
Inverse DCT, methods which down-sample a DCT Image or video by first 
converting it back to the spatial domain typically have a large computational 
complexity, A large computational complexity results in a longer calculation 
and a calculation which utilizes more processor time and povyer. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0005] Reference will now be made, by way of example, to the 
accompanying drawings which show an embodiment of the present 
application, and in which: 

[0006] Figure 1 shows a method of obtaining transforming matrices for 
down-sampling a DCT Image according to one aspect of the present 
application; 

[0007] Figure 2 shows a method of obtaining a training set in 
accordance with one aspect of the present application; 

[0008] Figure 3 shows a method of selecting transform matrices in 
accordance with one aspect of the present application; 

[0009] Figures 4A and 4B show a three layer network representation of 
the transform matrices; 

[0010] Figure 5 shows a method of training transform matrices using a 
learning with forgetting algorithm; 

[0011] Figure 6 shows a method of training transform matrices using a 
selective learning with forgetting algorithm; 

[0012] Figure 7 shows a method of obtaining a down-sampled DCT 
Image in accordance with one aspect of the present application; and 

[0013] Figure 8 shows an arrangement of electrical devices to be used 
In carrying out methods of the present application. 
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[0014] Similar reference numerals are used in different figures to 
denote similar components. 

DESCRIPTION OF SPECIFIC EMBODIMENTS 

[0015] In one embodiment of ttie present application, a metliod is 
provided for down-sampling a DCT image of size M x N directly in a OCT 
domain by applying two transform matrices. The first matrix, called a row 
transform matrix, Dim, is used to down-sample the number of rows in the DCT 
image. That is, it may be used to convert a DCT image, Cmn, with M rows into 
a DCT matrix with I rows. The number of columns in the row transform 
matrix is equal to the number of rows in the DCT image. The second matrix, 
called a column transform matrix, W^j, is used to down-sample the number of 
columns In the DCT image. That is, it may be used to convert a DCT Image 
with N columns into a DCT matrix with J columns. The number of rows In the 
column transform matrix is equal to the number of columns In the DCT 
image. Accordingly, the row transform matrix is of a size I x 1^1 and the 
column transform matrix is of size N x J. 

[0016] The DCT image, Cmn, may be any DCT image including, for 
example, a JPEG image. In some instances the DCT Image may represent a 
portion of a video, such as a frame of a video. For example, the DCT Image 
may be a single frame of a video of the DV video type. 

[0017] A down-sampled DCT Image is obtained using the transform 
matrices by multiplying the row transform matrix, Djm, with the product of 
the DCT image, Cmn, and the column transform matrix, Wf^j,. The down- 
sampled DCT image may be obtained using a transformation formula: 

^IM ^MN NJ 

[0018] Since one of the transform matrices Is applied to the DCT image 
on each side of the DCT image, the transform matrices may be called linear 
transform double sided matrices (LTDS). That is, one of the transform 
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matrices is applied to the left of the DCT image in the transformation formula 
and one Is applied to the right of the DCT image in the transformation 
formula. In this case, the row transform matrix Is applied to the left of the 
DCT image and the column transform maitrlx is applied to the right of the DCT 
Image. 

[0019] A wide range of spatial-domain down-sampling methods may be 
represented as a LTDS in the DCT domain. For example, in one embodiment, 
the transform matrices may be determined as: 



where Tss is a matrix of size S x S and which Is equal to: 

, where t is a DCT matrix. 



T = 



f[t] 0 0' 
0 0 
0 0 it]) 



E/M and Gnj are linear interpolation matrices for use in down-sampling in the 
spatial domain, and Amm is an M x M DFT transform matrix with its element 

given by a,, =-^expj^ ""'^^^ j,M = 0,1,,..,M -\,v = OX.,.,M -\ and A*^ is its 



conjugate* Similarly, Bnn is an N x N DFT transform matrix and 5^ is its 

conjugate matrix. Lmm and Rm are diagonal matrixes with diagonal elements 
being Lmi and R^n respectively. Lmj and R^^are two one-dimensional filters. 

[0020] The LTDS representation above may be obtained by 
concatenating the inverse DCT, spatial domain down-sampling of a certain 
format, and the DCT. More specifically, a spatial domain image Xmn niay be 
obtained by applying an inverse DCT to a DCT image Cmn- ^mn =^*°Ca«v°^ 
where t is a DCT matrix, t' is a transpose of the DCT matrix and □ denotes 
block-wise multiplications. Then, a spatial-domain method may be selected 
and used to down-sample Xmn to obtain an I x 3 Image, denoted as Xa.- 
Finally, the DCT may be applied to the I x J image, resulting in Vjj ^tax^jOt', 
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where IZ/j is a down -sampled DCT image. 

[0021] Generally, down-sampling in the spatial domain consists of two 
steps: low-pass filtering and interpolation. In some embodiments, a low pass 
filter may be implemented based on a 2D discrete Fourier transform (DFT). 

Given an image Xmn, the filtering output Xmm is obtained by: 

^^A^ = a^MM '^MN where Amm is an M x M DFT 

transform matrix with its element given by 



and Alfj^ Is its conjugate. Similariy, Bm is an N x N DFT transform matrix and 

is its conjugate matrix. Fmn is the low-pass filtering matrix in the DFT 
domain and the symbol ® denotes element wise multiplications, 

[0022] If Fmn is constructed using two one-dimensional filters (i.e. 

^A/jv =-^A/i '^/n) then the element wise multiplication may be removed, yielding 

'i^MM '^MN '^m)'^m '^m' where Uw and R^n are diagonal 
matrixes with diagonal elements being Lmj and Rj/v respectively. 

[0023] A linear interpolation filter may then be applied. The linear 
interpolation filter to be applied is in the form of interpolation matrices, Eim, 
Gnj. Therefore, the spatial domain down-sampled image may be computed as 

[0024] The concatenation of inverse DCT, spatial domain down sampling 
and DCT is therefore: 



[0025] The block wise multiplication can then be replaced by applying a 
result of 
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t o^^iMY =Tu^, Cj^, Where T^^,^ = 



-6 - . 

([t] 0 0 
0 ••. 0 

0 0 [t]) 



and t Is a OCT matrix. 



[0026] Consequently, a linear transform in the DCT domain can be 
obtained as: 

f'u -^iM ■W^\Nhere =T„_, -E^-A'^ -L^ -A^ -T^^, and 

[0027] That is, the row transform matrix may be represented as: 
^/M = T„j -E,^ A^,^ L^^ Aj^ Tmux- Similarly, the column transform matrix 
may be represented as =T^^ -B,, -R^ -B'^ G^, -T^^.. The row transform 
matrix and the column transform matrix may be obtained by determining a 
solution to the equations: =7),, -E,^ -A'^^ -L^^ -A,, -T,,, and 

[0028] Accordingly, In one aspect of the present application a method of 
down-sampling a DCT Image is provided. In one embodiment, the method 
includes steps of determining a row transform matrix using the equation: 
^iM =^/i., -EiM A^ A^^ TuM/ and determining a column transform 
matrix using the equation: ^T,,^ -B,, -R^ ^B'^ -G^ ..r„, 

[0029] As illustrated in Figure 1, in another embodiment of the present 
application, a method Is provided for obtaining transform matrices for down- 
sampling a DCT image of size M x N to a down-sampled DCT image of size I x 
J directly in the DCT domain. 

[0030] The method of Figure 1 will permit a row transform matrix D^ 
and a column transform matrix to be selected which jointly maximize the 
visual quality of down-sampled DCT Images that are obtained by applying the 
transform matrices to the DCT images while minimizing the computational 
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complexity of applying the transform matrices to the DCT images to obtain 
the down-sampled DCT images. That is, a row transform matrix and a 
column transform matrix are selected which provide a desired trade-off 
between computational complexity and visual quality. It is typically 
considered desirable to have a higher quality down-sampled DCT image and a 
lower computational complexity associated with obtaining that down-sampled 
DCT image. 

[0031] At step 102 optimization parameters may be obtained. The 
optimization parameters permit a user, system administrator, or programmer 
to have control over the balance between computational complexity and 
visual quality. For example, the optimization parameters may include a 
computational complexity trade-off parameter, p, which is used to balance 

the relative importance of various factors on the computational complexity. 
In some embodiments, the optimization parameters may include a rounding 
parameter, tj, which is used to determine how close a value must be to 
another value before the algorithm will attempt to round to the other value. 
The optimization parameters may also include a quality and complexity trade- 
off parameter A which is used to adjust the desired trade-off between the 
visual quality and complexity. That is, the quality and complexity trade-off 
parameter A may be used to specify whether optimization of the transform 
matrices will be more or less focused on visual quality or complexity. The 
optimization parameters may also include selective learning with forgetting 
threshold parameters, Wo, do,, which will be used in a selective learning with 
forgetting stage to adjust the threshold above which an element in the 
transform matrices will be considered protected and not pushed towards zero. 

In some embodiments, the optimization parameters may include a learning 
step size parameter, a , which is used to adjust the step size of learning 
operations. That is, it is used to adjust how great the degree with which the 
transform matrices will be adjusted after an iteration of learning. 

[0032] Depending on the method used to optimize the transform 
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matrices, some or all of the optimization parameters discussed above may be 
obtained in step 102. In some embodiments, these parameters are stored in 
a memory and they are retrieved from the memory. 

[0033] In other embodiments, the optimization parahieters may be 
obtained from a user. A user may be prompted to enter desired optimization 
parameters using an input device. In some embodiments, a setup module 
may be run by the user which allows the user to enter optimization 
parameters. After the user enters the optimization parameters, they may be 
saved to the memory. 

[0034] In some embodiments, the optimization parameters are entered 
on a server which obtains the transform matrices. 

[0035] Next, at step 104, a training set is obtained which will be used in 
training the transform matrices. The training set Includes at least one DCT 
reference image, Cmn, and typically includes a corresponding down-sampled 
DCT reference image, Vu, for each reference image. The DCT reference 
image Is of the size MxN and the down-sampled DCT reference image Is of the 
size IxJ. 

[0036] As will be described more fully below, the training set will be 
used to evaluate the quality of down-sampled images obtained using the 
transform matrices relative to the quality of the down-sampled DCT reference 
image. 

[0037] Typically, the training set will consist of more than one DCT 
reference image Cmn and more than one down-sampled DCT reference image, 
Vu. The down-sampled DCT reference Image Vuis obtained using another 
down-sampling method. Typically, the down-sampled DCT reference image 
Vu is obtained using a spatial domain reference image in which the DCT 
reference image Cmn is converted to the spatial domain using an inverse DCT 
transform, and the spatial domain image is down-sampled using a spatial 
domain down-sampling method. Then, the down-sampled spatial domain 
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reference image is transferred back to the DCT domain using a DCT transform 
to obtain the down-sampled DCT reference image, Vd. 

[0038] This will permit the transform matrices to be trained multiple 
times, thus increasing the accuracy of the transform matrices. For example, 
in some embodiments the training set Includes five DCT reference images Cmn 
and five down sampled DCT reference Images, Vu, each corresponding to one 
of the DCT reference images. 

[0039] The step 104 of obtaining a training set may include a step of 
applying a pre-selected down sampling method to a DCT reference image to 
obtain the down-sampled DCT reference images. Typically, the pre-selected 
down-sampling method is a spatial domain down-sampling method. 

[0040] Referring now to Figure 2, a method is shown for obtaining a 
training set using a spatial domain down-sampling method. At step 202 a 
down-sampling method Is selected. Typically, the selected down-sampling 
method will be a down-sampling method which produces a down-sampled 
DCT reference image with an optimal visual quality since the visual quality of 
the down-sampled image produced according to the present method is 
dependent on the visual quality of the down-sampled DCT reference image. 

[0041] In some instances, the down-sampling method is pre- 
determined. 

[0042] Where the down-sampling method is a spatial domain down- 
sampling method, the method of obtaining the training set includes a step 
204 of converting at least one of the DCT reference images to the spatial 
domain by applying an inverse DCT to produce a spatial domain reference 
Image. Next, at step 206, the spatial domain reference image is down- 
sampled using the selected spatial domain down-sampling method to produce 
a spatial domain down-sampled reference image. Then, at step 208, the 
spatial domain down-sampled reference image is converted back to the DCT 
domain to obtain the down-sampled DCT reference image. 
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[0043] Typically, the step 206 of down-sampling the converted DCT 
reference image in the spatial domain includes the steps of: 1) applying a low 
pass filter to the spatial domain reference Image to produce a filtered Image; 
and 2) interpolating within the filtered image to produce the spatial domain 
down-sampled reference image. 

[0044] With respect to the step of filtering, a filter is typically selected 
which provides a good trade-off among three factors: the reduction or 
elimination of aliasing, the preservation of low-frequency components, and 
the reduction or elimination of ringing. Generally, a filter with a sharp 
transition band provides a good performance on anti-aliasing and preserving 
low-frequency components, yet a sharp transition band incurs ringing along 
intensity edges in the filtered Image. 

[0045] In some embodiments, a Butterworth filter is used. The 
Butterworth filter provides a good trade-off among the three factors (aliasing, 
low-frequency components, and ringing). That is, two one dimensional 
Butterworth filters may be selected with the frequency response function: 

where is the cutoff frequency and L represents the order of the filter which 

characterizes the transition band. In some embodiments, the filter may be a 
tenth order filter (i.e. L=10). 

[0046] As noted above, this filter may be applied to the spatial domain 
reference image to obtain a filtered image. 

[0047] Following the step of low-pass filtering, interpolation may be 
performed on the filtered image to obtain the spatial domain down-sampled 
reference image. Interpolation Involves estimating the values of the spatial 
domain down-sampled reference image at certain points. 

[0048] A wide range of interpolation methods may be used. For 
example, nearest neighbour interpolation or spline interpolation may be used. 
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The Nyquist-Shannon interpolation uses a sine function which achieves 
theoretically optimal performance. 

[0049] In some embodiments, cubic interpolation may be used which 
provides a smooth function with a continuous first derivative. While the 
nearest neighbor interpolation corresponds to a square interpolation function, 
the bicubic interpolation is obtained by convolving three square functions. 

[0050] It will be apparent to a person skilled in the art that, while 
specific spatial domain down-sampling methods have been discussed above, 
the method discussed herein may be used with any other spatial domain 
method. 

[0051] The training set may be stored in a memory after it is created. 
In some embodiments, the memory may contain a database of training sets 
for various down-sampling ratio. For example, the database may contain 
data training sets to train transform matrices to convert a matrix of size M x 
N to a matrix of any one of a number of various sizes (i.e. I x J, O x P, Q x R, 
etc.). 

[0052] In embodiments where training sets are previously generated 
and stored In memory, the step 104 of obtaining a training set may merely 
require the training set to be retrieved from the memory. 

[0053] Once a training set Is obtained, the method of obtaining 
transform matrices for down-sampling a DCT image includes a step 106 of 
selecting transform matrices that minimize an optimization problem. The 
optimization problem is a function of the error between the down-sampled 
image obtained by applying the transfornn matrices to the reference Image 
and the down-sampled DCT reference image. The optimization problem is 
also a function of the computational complexity of applying the transform 
matrices. 

[0054] It will be appreciated that various methods may be employed to 
find transform matrices which jointly optimize the visual quality and 
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computational complexity. In some embodiments, discussed more fully 
below, a multi-layer network may be used to find transform matrices which 
solve the optimization problem. 

[0055] Accordingly, in step 106, transform matrices are found which 
obtain a suitable balance between maximizing the quality of the down- 
sampled image obtained using the matrices, while minimizing the 
computational complexity associated with down-sampling using the transform 
matrices. That is, ah LTDS is found with the best trade-off between the 
fidelity of a down-sampled image and the computational complexity of the 
using the LTDS in the sense of minimizing the joint cost. 

[0056] In order to select the transform matrices which minimize the 
joint cost, a value representing the visual quality of the down-sampled DCT 
images obtained using the transform matrices may be determined. A value 
representing the computational complexity associated with applying the 
transform matrices to a DCT image in order to down-sample the DCT image 
may also be determined. The joint cost of the transform matrices may then 
be determined as a function of the value representing the visual quality of the 
down-sampled DCT images obtained using the transform matrices and the 
computational complexity associated with applying the transform matrices to 
a DCT image in order to down-sample the DCT image. 

[0057] The visual quality of the down-sampled Image is measured 
relative to the quality of the down-sampled DCT reference image obtained 
using the other down-sampling method. Where quality is measured in this 
manner, the linear transform double sided matrices. Dim and Wnj, may be 
selected to minimize the difference in between the quality of the image 
obtained using the transform matrices (i.e. C^^ -Wj^j) and the quality of a 
down-sampled DCT reference image, Vu, obtained using the other down- 
sampling method. That Is, the transform matrices, Dim and Wnj, are selected 
to satisfy the following optimization problem: 
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rnin (ID,^ • C^. W,,-V^f+ A,- r^) 

where D^j^ '^ac/ 's a down-sampled image obtained using the transform 
matrices, V^j is the DCT reference Image which represents a down-sampled 
image obtained using another down-sampling method, is a measure of the 
computational complexity associated with using the transforni matrices 
^iM^^Nj to down-sample an image Cmn and, A, is a quality and complexity 
trade-off parameter. 

[0058] The quality and complexity trade-off parameter, X , may be used 
to balance the trade-off between the visual quality and computational 
complexity. That is, the quality complexity trade-off parameter may be used 
to adjust whether optimization will be more or less focused on visual quality 
or complexity. 

[0059] As used in the above equation, the quality and complexity trade- 
off parameter, A , Is applied to the computational complexity associated with 
using the transform matrices. In this case, a greater quality complexity trade- 
off parameter, X , will result in more importance being placed on complexity. 
Where the quality complexity trade-off parameter, A, is applied in this 
manner, an example of a suitable value may be 0.1. However, it will be 
appreciated that various values for the quality and complexity trade-off 
parameter, X, may also be used. 

[0060] Where a quality and complexity trade-off parameter, X , is used, 
the method may further include a step of retrieving the quality and 
complexity trade-off parameter. This step may be Included in the step 102 of 
obtaining the optimization parameters. The quality complexity trade-off 
parameter, X, may be retrieved from a memory. In other embodiments, the 
a quality and complexity trade-off parameter, X, may be obtained via user 
Input. 
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[0061] The computational complexity may be a function of the absolute 
values of elements in the row-size transform matrix and the column 
transform matrix. That is, in some embodiments, the computational 
complexity may be calculated as: = \Djj^\ + \W^\, where (j defines the k norm 
of a matrix. 

[0062] The computational complexity may, therefore, be a function of 
how far the elements of the matrices are from zero. It will be appreciated 
that the computational complexity of the transform matrices will be minimized 
when all elements of the transform matrices are zero. Of course, it is 
undesirable to have transform matrices where all elements are zero since it 
would not produce a down-sampled image and the quality, would therefore, 
be undesirable. 

[0063] To determine how far the elements of the matrices are from 
zero, the method may include a step of calculating the summation of the 
absolute values of elements in the transform matrices. 

[0064] In some embodiments, the complexity model is established to 
ignore the non-zero effect of matrix elements with values greater than a 
threshold Wo, do on computational complexity. For example, the complexity 
model may be: 

[0065] That is, the method may comprise a step of calculating the 
summation of the absolute value of all elements In the row transform matrix 
and the column size matrix that are below two thresholds Wo, do. In many 
cases, the thresholds may be the same (i.e. Wo= do). 

[0066] Besides the number of non-zero elements in the row transform 
matrix and the column transform matrix, the complexity for computing a 
down-sampled DCT image using the transform matrices is also related to how 
multiplications may be Implemented. In general, a multiplication may be 
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approximated by a series of additions and sliifts. 

[0067] In some embodiments, the computational complexity may be 
related to tiie number of non-zero elements in the binary representation of 
the number That is, it Is considered more desirable to have a zero in a 
binary representation than a one. Accordingly, the computational complexity 
may evaluate the number of non-zero elements in the binary representation 
of each of the row transform matrix and the column transform matrix. This 
may be done by locating a computationally more desirable value within the 
neighborhood of a current element of the transform matrix. The 
computational complexity of the value may be a function of the difference 
between the current element of the transform matrix and the computationally 
more desirable value. 

[0068] More particularly, for any element, b, in one of the transform 
matrices, a set of binary digits, {a,}, may be found which satisfies the 
equation: 

{a,} = arg mm ZKI 

where n is a rounding parameter, which is used to adjust how large the 
neighborhood of an element, may be. That Is, n Is used to adjust how 
close the element, b, must be to another value having more zeros in its 
binary representation before the other value will be considered desirable. The 
binary representation may be converted bacic into a decimal number system 
representation to obtain a quantized decimal value. The procedure of locating 
a neighboring value which is less complex than the current element, b, may 
be referred to as a quantization procedure, Q(fc). 

[0069] In some embodiments, a binary number system is used In which 
every binary digit may be signed. That Is, the binary digits may be an 
element of the set consisting of positive one (1), zero (0), or minus one (-1). 
(i.e. a,, e {1,-1,0)). In some embodiments, such a binary representation may 
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be used since it permits binary representations with a maximum number of 
zeros (0). For example, if we consider the binary representation of the 
number 0.46875 in a binary system in which each element must be selected 
from the set consisting of only one (1) and zero (0), the binary representation 
would be 0.01111. That is, 

0.46875 = (0 X 2°) + (0 x 2'') + (1 x 2"^) + (1 x Z^) + (1 x 2-^) + (1 x 2'^) 

Using this number system, four ones are needed to represent the number. 

In contrast, the same number could be represented using the signed system 
mentioned above using only two ones: 

0.46875=(0 X 2^)+(l x 2'^)+(0 x r^)+(0 x 2'3)+(0 x 2-^)+((-l) x 2'^). 

[0070] Typically, the magnitude of ail elements in the transform 
matrices Dfj^,W^j is In the range [0, 8). That Is, the magnitudes of all 

elements is greater than or equal to 0 and less than 8. If this assumption is 
made, then the integer portion of the elements may be represented with 
three binary digits (i.e. 2^ 2^ 2°). The number of binary digits used to 
represent the decimal fraction portion of the elements may depend on the 
desired visual quality of down-sampled DCT images obtained using the 
matrices. For example, if fewer binary digits are used to represent the 
decimal fraction portion of the elements, the computational complexity 
associated with using the transform matrices will be reduced, but the visual 
quality of the down-sampled DCT Images may also be reduced. In some 
embodiments, fifteen (15) binary digits are allocated for representing the 
decimal fraction portion of the elements of the transform matrices. 

[0071] If the binary system discussed above is used, the quantization 
procedure for an element, b, may be represented as: 



[0072] That is, the quantization procedure referred to above determines 
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a set of binary numbers, {ai} for a computationally more desirable value. 
Then the decimal representation of the computationally more desirable value 
is determined. 

[0073] The difference between the current element, b, of the transform 
matrix and the computationally more desirable value may then be found. For 
example, the computational complexity associated with the number of non- 
zero elements In the binary representation of the elements of the transform 
matrices may be determined as: 

where Q(Dim), Q(Wnj) mean to apply a quantization procedure to each 
element of the row transform matrix and the column transform matrix 
respectively. 

[0074] Since the computational complexity is a function of both the 
number of non-zero decimal representations in the transform matrices and 
the number of non-zero elements In the binary representation of the elements 
of the transform matrices, the computational complexity may determined as: 

where p is a computational complexity tradeoff parameter used to balance 
the relative importance on the computational complexity of the number of 
non-zero elements in the transform matrices and the difference between 
current weights in the transform matrix and weights that are close to the 
current weights but which would have more zeros in their binary 
representation. In some embodiments, p is equal to 0.5- 

[0075] Referring now to Figure 3, a method of obtaining transform 
matrices according to one aspect of the present application Is shown. In this 
embodiment, a multiple-layer networl< is used to find transform matrices 
which minimize the joint cost of the transform matrices. That is, a multiple- 
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layer network is used to find transform matrices which offer a desired baiance 
between computational complexity and visual quality. 

[0076] At step 302, transform matrices are Initialized. Referring to 
Figures 4A and 4B a multiple-layer network representation of the transform 
matrices is shown. A three layer network structure is used, in which there is 
an input layer 402, representing an input image Cmn of size M x N. There Is 
also a hidden layer 404, representing an intermediary matrix Yin of size I x N. 
The multiple-layer network structure also contains an output layer 405, 
representing a down-sampled image Zu of size I x J. Connections 408, 410 
are selectively built up among units in each two layers to simulate the matrix 
multiplication operation in the linear transform. That is, connections 408 
between the Input layer 402 and the hidden layer 404 represent the 
application of the row transform matrix Dim to the input image Cmn- Similarly, 
connections 410 between the hidden layer 404 and the output layer 406 
represent the application of the column transform matrix Wnj to the 
intermediary matrix Yin. 

[0077] Connections between the layers are established according to a 
number of rules. With respect to the connections 408 between the input 
layer and the hidden layer, connections are established from units in a given 
column (i.e. nj of the input layer to units of the same column (i.e. nO in the 
hidden layer. That is, the input layer and the hidden layer have the same 
number of columns. Units in a given column of the input layer are fully 
connected to units in the same column of the hidden layer. Valid connections 
between any of two columns of the input layer and the hidden layer share the 
same weight matrix i.e. Dim* 

[0078] Similariy, connections 410 between the hidden layer 404 and the 
output layer 406 are built up among rows. Connections are established from 
units in a given column (I.e. ii) of the hidden layer 404 to units in the same 
column of the output layer 406 (i.e. ii). Valid connections 410 between any 
two rows of the hidden layer and the output layer share the same weight 
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matrix i.e. Wnj. 

[0079] Accordingly, the output of the hidden layer 404 is computed as 
= • and the output of the output layer 406 is computed as 

[0080] Referring again to Figure 3, during the step of initialization, an 
initial starting weight may be placed In the elements of the transform 
matrices Di„, Wnj,. The Initial starting weight represents a starting point from 
which a decay procedure will be applied. In some embodiments, random 
weights may be placed in the elements of the transform matrices Dim, Wnj. 

[0081] At step 304, a learning with forgetting algorithm is applied to the 
transform matrices to obtain transform matrices with a reduced learning 
objective function. This stage Is used to find a skeleton structure of the 
transform matrices. 

[0082] It will be appreciated that due to a random initialization of the 
connection weights, some redundant connections may possess an initial 
weight with a large value. Accordingly, the learning with forgetting stage will 
apply a constant decay to all elements In order to remove redundant 
connections which are present due to a random initialization of the weights. 
Such redundant connections are removed as much as possible. This approach 
leads to a constant decay for all non-zero elements, forcing as many elements 
to be zero as possible. 

[0083] The learning objective function at this stage is: 

where is the learning objective function, Z);^ is the row transform 
matrix, Wf^j is the column transform matrix, C^^^ is the DCT Image, 

•W^ is the down-sampled DCT image obtained using tlie transform 
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matrices, V,j is a down-sampled DCT reference image obtained using anotlier 
down-sampling method, is the computational complexity associated with 
using the transform matrices Dj^.W^^j to down-sample an image Cmn and X is 

a quality and complexity trade-off parameter used to balance a trade-off 
between visual quality of down-sampled DCT images and the computational 
complexity of obtaining the down-sampled DCT images. In this case, the 
computational complexity Is a function of the absolute value of all elements of 
the transform matrices. That is, =|^/j»#| + |^Ary|/ where ||deifines the k 

norm of a matrix. 

[0084] Since the complexity for this stage is a function of the absolute 
value of all elements in the matrices, the complexity is minimized If every 
element is zero. 

[0085] It will also be noted that the visual quality at this stage is 
measured as the mean square error of the visual quality of the down-sampled 
DCT image produced using the transform matrices and the visual quality of 
the down-sampled DCT image produced by applying another down-sampling 
method to the same initial image. 

[0086] As illustrated in Figure 5, the learning with forgetting algorithm 
includes a step of adjusting matrices by a learning amount. The learning 
amount may be calculated by using a back propagation algorithm. 

[0087] First, the input signal is passed forward to compute network 
outputs: 

[0088] A network error is then determined by comparing the network 
output to the output for the down-sampled DCT reference image, which was 
produced using another down-sampling domain method. The network error is 
then propagated backward. 
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= ^z, - Vfj => (AJO^ = (AZ)^ . iW')j, 

[0089] The learning announts may then be deternnined for each of the 
transform matrices. The learning amounts may be determined as: 



where sgn(x) is the sign function as: 

fl 



sgn(x) = 



-,jc>0 
2 

0,x = 0 
2 



[0090] The learning amounts may then be applied to the transform 
matrices to obtain adjusted transform matrices: 

where the superscripts (n) and (n+1) accord to the nth and (n+l)th iterations 
and a is a small positive number named the learning factor or learning step 
size parameter. The learning factor is used to adjust the step size of learning 
operations. That is, it is used to adjust how great the adjustment will be with 
each iteration of learning. In some embodiments, the learning factor is 1x10" 

6 

[0091] Typically, there will be multiple learning with forgetting 
iterations. In one embodiment, at step 504, a determination is made 
regarding whether the decrement of the learning objective function was less 
than a predetermined threshold. If It was not, another iteration of learning 
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with forgetting will be performed. If it was less than the predetermined 
threshold, then the learning with forgetting stage will be considered to be 
completed. That is, the learning with forgetting stage is complete when each 
iteration of learning no longer yields a sufficient improvement to the 
transform matrices in terms of minimizing the joint cost. 

[0092] In some embodiments, the predetermined threshold is zero. 
That is, the learning objective function will be completed when the joint cost 
of the transform matrices is no longer Improving with each iteration. 

[0093] Learning with forgetting normally ends with a skeleton structure 
but a large distortion of • C • - vf . That Is, a constant decay to elements 
with large values will introduce a large distortion to the visual quality, 
measured as ||;r,^ -T'\Dj^ •^Ay)f . where Xy is a spatial domain down- 
sampled image and T\x) is an inverse DCT transform. 

[0094] As illustrated in Figure 3, a step 306 of performing a selective 
learning with forgetting algorithm may then be used to tune the structure for 

better trade-off between the distortion of \D'C'W -vf an6 the complexity. 

The selective learning with forgetting stage protects certain large elements 
from decay so that they can be trained to focus on providing better visual 
quality. 

[0095] Since the selective learning with forgetting stage protects certain 
large elements from decay, the complexity model may include at least one 
selective learning with forgetting threshold parameter (Wo or do) to be used 
for determining what is or is not considered a large element. In some 
embodiments, each of the transform matrices has its own selective learning 
with forgetting threshold parameter (Wo or do). In other embodiments, the 
selective learning with forgetting threshold parameter (Wo or do) is the same 
for both transform matrices. 

[0096] The selective learning with forgetting threshold parameter (Wo or 
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do) has the effect of adjusting the trade-off between visual quality and 
complexity since a higher threshold may result In a higher quality image, but 
will result in less elements of the transform matrices which have been pushed 
toward zero. In some embodiments, the selective learning with forgetting 
threshold parameter (Wo or do) is 0.1. 

[0097] The learning objective function at this stage is very similar to the 
learning objective at the learning with forgetting stage: 



where is the learning objective function, i),^ is the row transform 
matrix, W^^j is the column transform matrix, is the DCT image, 
Am 'Cf^ff'Wf^j is the down-sampled DCT image obtained using the transform 
matrices, Vjj is a down-sampled DCT reference image obtained using another 
down-sampling method, Is selective learning with forgetting computational 
complexity associated with using the transform matrices Djj^.Wj^j to down- 
sample an image Cmn and A is a quality and complexity trade-off parameter 
used to balance a trade-off between visual quality of down-sampled DCT 
images and the computational complexity of obtaining the down-sampled DCT 
images. 

[0098] In this case, however, the complexity model is established to 
ignore the non-zero effect of matrix elements with values greater than the 
threshold on the complexity. For example, the complexity model for the 
learning with forgetting stage may be: 



[0099] That Is, the method may comprise a step of calculating the 
summation of the absolute value of all elements in the row transform matrix 
and the column size matrix that are below two thresholds Wo, do. In many 



\\dJ<do 
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cases, the thresholds may be the same (i.e. Wo= do). 

[00100] As discussed previously; besides the number of non-zero 
elements In the row transform matrix and the column transform matrix, the 
complexity for computing a down-sampled DCT (mage using the transform 
matrices is also related to how multiplications may be implemented. The 
computational complexity may be related to the number of non-zero elements 
in the binary representation of the number. That is, it Is considered more 
desirable to have a zero in a binary representation than a one. Accordingly, 
the computational complexity may evaluate the number of non-zero elements 
In the binary representation of each of the row transform matrix and the 
column transform matrix using the algorithm discussed above. This may be 
done by locating a computationally more desirable value within the 
neighborhood of a current element of the transform matrix. The 
computational complexity of the value may be a function of the difference 
between the current element of the transform matrix and the computationally 
more desirable value. 

[00101] l^ore particularly, for any element, b, in one of the transform 
matrices, a set of binary digits, {a/}, may be found which satisfies the 
equation: 



where n is a rounding parameter, which is used to adjust how large the 
neighborhood of an element, b, may be. That is, n is used to adjust how 
close the element, b, must be to another value having more zeros in its 
binary representation before the other value will be considered desirable. The 
binary representation may be converted back into a decimal number system 
representation to obtain a quantized decimal value. The procedure of locating 
a neighboring value which is less complex than the current element, b, may 
be referred to as a quantization procedure, Q(fe). 
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[00102] If the binary system discussed above is used, the quantization 
procedure for an element, 6, may be represented as: 

e(6) = 'f a, .2'",fl, E{1,-1,0), where {a,} = arg min IK| 

[00103] That is, the quantization procedure referred to above determines 
a set of binary numbers, <ai} for a computationaily more desirable value that 
is within the neighborhood of the element, b. Then the decimal 
representation of the computationally more desirable value Is determined. 

[00104] The difference between the current element, b, of the transform 
matrix and the computationally more desirable value may then be found. For 
example, the computational complexity associated with the number of non- 
zero elements in the binary representation of the elements of the transform 
matrices may be determined as: 

where Q(Dim), QCWnj) mean to apply a quantization procedure to each 
element of the row transform matrix and the column transform matrix 
respectively. 

[00105] Since the computational complexity is a function of both the 
number of non-zero decimal representations in the transform matrices and 
the number of non-zero elements in the binary representation of the elements 
of the transform matrices, the computational complexity in the selective 
learning with forgetting algorithm may determined as: 

where/? is a computational complexity tradeoff parameter used to balance 
the relative importance on the computational complexity of the number of 
non-zero elements in the transform matrices and the difference between 
current weights In the transform matrix and weights that are close to the 
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current weights but which would have more zeros in their binary 
representation. • In some embodiments, p is equal to 0.5. 

[00106] The quantization procedure discussed above, which examines 
whether there Is a neighboring value which is computationally less complex, Is 
generally applied at the selective learning with forgetting stage because its 
corresponding contribution to the complexity function is at a level similar to 

=I^/^'Lmo +K^Ik|<h. which is much less than ^\D,^\^^^j\. 

[00107] Note, however, that regardless of which stage the quantization 
procedure is applied at, the computational complexity tradeoff parameter p 

may be used to balance the relative Importance on the computational 
complexity of the number of non-zero elements In the transform matrices and 
the difference between current weights in the transform matrix and weights 
that are close to the current weights but which would have more zeros In 
their binary representation. 

[00108] Referring now to Figure 6, a method of applying a selective 
learning with forgetting algorithm to the transform matrices is shown. At step 
602, the transform matrices are adjusted by a learning amount. 

[00109] The learning amount is determined in much the same way as the 
learning amount is determined for the learning with forgetting stage. 

[00110] First, the input signal is passed forward to compute network 
outputs: 

[00111] A network error Is then determined by comparing the network 
output to the output for the down- sampled DCT reference image, which was 
produced using another down-sampling domain method- The network error is 
then propagated backward. 
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[00112] The learning amounts may then be determined for each of the 
transform matrices. The formula for determining the learning amounts in the 
selective learning with forgetting stage differs from the formula for 
determining the learning amounts for the learning with forgetting stage. In 
the selective learning with forgetting stage, the learning amounts may be 
determined as: 



where: 



,0>x>O 



O,x = O;x>0;x<-0 

--,'9<x<0 

. 2' 



sgn(^) = 



-,x>0 
2 

0,x = 0 

--,;c<0 
2 



and A is the quality and complexity trade-off parameter and'^ is the 
computational complexity tradeoff parameter. 

[00113] As before with the learning with forgetting algorithm, the 
learning amounts may then be applied to the transform matrices to obtain 
adjusted transform matrices: 



NJ 



where the superscripts (n) and (n+1) accord to the nth and (n+l)th iterations 
and a is a small positive number named the learning factor or learning step 
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size parameter. As before, the learning factor is used to adjust the step size 
of learning operations. The learning factor for the selective learning with 
forgetting stage may be a different value than the learning factor for the 
learning with forgetting stage. 

[00114] Typically, there will be multiple selective learning with forgetting 
iterations. In one embodiment, at step 604, a determination is made 
regarding whether the decrement of the learning objective function was less 
than a predetermined threshold. If it was not, another iteration of learning 
with forgetting will be performed. If It was less than the predetermined 
threshold, then the selective learning with forgetting stage will be considered 
to be completed. That is, the selective learning with forgetting stage is 
complete when each iteration of learning no longer yields a sufficient 
Improvement to the transform matrices in terms of minimizing the joint cost. 

[00115] In some embodiments, the predetermined threshold is zero. 
That is, the learning objective function will be completed when the joint cost 
of the transform matrices is no longer improving with each iteration. 

[00116] Following the selective learning with forgetting algorithm, in 
some embodiments the computational complexity of the transform matrices 
may be further reduced by combining arithmetic operations. 

[00117] Referring now to Figure 7, according to one aspect of this 
application, a method is provided for down-sampling a DCT Image of size M x 
N in the DCT domain to produce a down-sampled Image of size I x J, 

[00118] In step 702, transform matrices are obtained. The transform 
matrices include a row transform matrix of size I x M and a column transform 
matrix of size N x J. The transform matrices may have been produced 
according to any of the methods of producing transform matrices for down- 
sampling a DCT Image directly in the DCT domain outlined above. That is, 
the row transform matrix and the column transform matrix are matrices that 
have been selected to jointly optimize the visual quality of down-sampled DCT 
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images obtained using the matrices and minimize the computational 
complexity associated with applying the matrices to DCT images. The visual 
quality of the down-sampled images is determined with reference to the 
visual quality of another down-sampled image obtained using another down- 
sampling method. In some embodiments, the matrices have been 
determined using a multiple-layer networl< to solve an optimization problem. 

[00119] In some embodiments, the step 702 of obtaining includes 
looking up the row transform matrix and the column transform matrix in a 
memory. In other embodiments, the step 702 of obtaining includes loolcing 
up appropriate transform matrices in a database of transform matrices. In 
such embodiments, the database may Include transform matrices for various 
down-sampling ratios and the step of obtaining Includes selecting the 
transform matrices with a desired down-sampling characteristic, such as a 
desired down-sampling ratio. 

[00120] In some embodiments, the database may be stored remotely on 
a server. The step of obtaining includes a step of requesting appropriate 
transform matrices from the server and receiving the row transform matrix 
and column transform matrix from the server. To ensure that the correct 
transform matrices are obtained, the step of requesting may include 
transmitting system identification data to the server. For example, the 
identification data may include data identifying a display resolution of a device 
which will be used to display the down-sampled image and/or data identifying 
processor speed information of a processor which will be used to transform 
the DCT image into the down-sampled DCT image. 

[00121] In this way, transform matrices may be received which are best 
suited for the device's display or processing capabilities. 

[00122] Following the step of obtaining, a step 704 of determining an 
intermediary matrix as the product of the DCT image and one of either the 
row transform matrix or the column transform matrix. Since matrix 
multiplication Is not commutative, the arrangement of multiplication may be 
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important. The row-transform matrix, Dim, is generally to the left of the DCT 
Image, Cmn, in the transformation formula and the column-transform image, 
Wnj, is generally to the right of the DCT image in the transformation formula. 

[00123] That is, since • C^j^ • Wffj W^j • C^u Am / the latter expression 

is not used. However, since matrix multiplication is associative, the order 
may be modified to some extent. The row size matrix may be multiplied with 
the DCT image and the result multiplied with the column transform matrix 
(i.e. (p^^ 'CMf/)'^m)f or alternatively the DCT image may be multiplied with 
the column transform matrix and the result multiplied with the row transform 
matrix (i.e. D,^ -{Cj,^ W^j)). 

[00124] Accordingly, in step 704 an intermediary matrix, K, is obtained 
as the product of the DCT image and one of either the row transform matrix 
or the column transform matrix is obtained as either: K={Djj^ • C^^) or 

[00125] Next, at step 706, the down-sampled DCT image is obtained as 
the product of the intermediary matrix K and the transform matrix that is not 
used in the previous step of determining (either Dim or W/vj). For example, if 
the intermediary matrix K is determined as K-{Djj^ C^), then the down- 
sampled DCT image Is obtained as Xu=(A: ^f^^). Alternatively, if the 
intermediary matrix K is determined as K=(Cji,^ 'W^j).tY\&r\ the down-sampled 
DCT Image is obtained as Xn= {Dj^ K). 

[00126] It will be appreciated that transcoders for down-sampling images 
are used in wide variety of electrical devices. Accordingly, the methods 
described above may be Implemented on any suitable electronic device 
comprised of a processor and a memory. 

[00127] By way of example, in one embodiment, illustrated in FIG. 8, the 
methods described above are used to down-sample images to a size suitable 
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for display on a display of a mobile device 802. The mobile device 802 is 
connected to a server 804. The mobile device 802 may be connected to the 
server 804 in any one of a number of methods. For example, the mobile 
device 802 may be connected to the server through a cellular network, a 
wireless wide area network (WWAN), or a wireless local area network (WLAN). 
It will be appreciated that other methods of connection, both wired and 
wireless, may also be used. 

[00128] The server 804 may be an Internet gateway which provides 
access to the Internet. In some embodiments, the server 804 includes a 
transcoder 808 which is configured to down-sample images from a size M x N 
DCT image to a size I x J DCT image. In some embodiments, the M x N DCT 
images are images received from the Internet. 

[00129] The transcoder 808 comprises a processor 810 and a memory 
812. The processor 810 may be configured to execute the steps of any of the 
preceding methods to find transform matrices for down-sampling a DCT 
image directly in the DCT domain. The processor 810 may also be configured 
to down-sample a DCT image directly In the DCT domain using the transform 
matrices. In one embodiment, the memory 812 Is used to store transform 
matrices. The memory 812 may also be used to store any of the optimization 
parameters. 

[00130] The processor 810 is connected to the memory 812. In one 
embodiment, the processor is configured to select a spatial domain down- 
sampling method and to apply the selected spatial domain down-sampling 
method to a DCT image to produce a DCT reference image. The processor 
810 may be configured to store optimization parameters which have been 
found to minimize or reduce the optimization problem to the memory 812. 

[00131] The processor 810 may be configured to determine or further 
optimize transform matrices during periods when the processor is 
experiencing little use. 
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[00132] In some embodiments, the processor 810 is configured to 
produce a down-sampled Image of size I x J according to tlie method 
illustrated in Figure 7 and described above. 

[00133] In some embodiments, the transcoder is included on the mobile 
device 802. The mobile device 802 also has a display for displaying images 
and a communication module for communicating with the server. The mobile 
device 802 may also include a processor connected to the communication 
module. The processor is configured to execute the down-sampling method 
illustrated In Figure 7. That Is, the processor is configured to obtain from the 
server the transform matrices and to determine an intermediary matrix as the 
product of the DCT image and one of either the row transform matrix or the 
column transform matrix and to determine a down-sampled DCT image of 
size IxJ as the product of the intermediary matrix and the transform matrix 
whfch was not used in determining the intermediary matrix. This 
embodiment, in which the server determines and provides the transform 
matrices to the mobile device 802 which then performs multiplication 
operations using the transform matrices may be particularly useful for down- 
sampling Images which are locally produced on the mobile device using, for 
example, a camera. This configuration permits a user to down-sample a 
locally produced image obtained using the camera to minimize the bandwidth 
used in transmitting the Image over the networlc. Accordingly, In some 
embodiments, the mobile electronic device 802 includes a camera. 

[00134] Accordingly, in one aspect the present application provides a 
method of down-sampling a DCT image of size M x N In the DCT domain to 
produce a down-sampled image of size 1x3 comprising steps of: a) obtaining 
transform matrices comprising a row transform matrix of size I x M and a 
column transform matrix of size N x J; b) determining an intermediary matrix 
as the product of the DCT image and one of either the row transform matrix 
or the column transform matrix; and c) determining the down-sampled DCT 
image as the product of the intermediary matrix and the transform matrix not 
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used in the previous step of determining. 

[00135] In another aspect, the present application provides a transcoder 
for transcoding a DCT Image. The transcoder has a memory having 
transform matrices stored thereon. The transform matrices comprise a row 
transform matrix of size I x M and a column transform matrix of size N x J. 
The transcoder also includes a processor connected to the memory and 
configured to obtain from the memory the transform matrices, and to 
determine an Intermediary matrix as the product of the DCT image and one 
of either the row transform matrix or the column transform matrix. The 
processor Is also configured to determine a down-sampled DCT image of size 
I X 3 as the product of the intermediary matrix and the transform matrix not 
used In determining intermediary matrix. 

[00136] In another aspect, the present application provides a mobile 
electronic device having a transcoder for down-sampling a DCT image. The 
mobile electronic device includes a display for displaying images. The mobile 
electronic device includes a communication module for communicating with a 
server and a processor connected to the communication module. The 
processor is configured to obtain from the server the transform matrices 
including a row transform matrix and a column transform matrix. The 
processor Is also configured to determine an intermediary matrix as the 
product of the DCT image and one of either the row transform matrix or the 
column transform matrix. The processor is also configured to determine a 
down-sampled DCT image of size I x J as the product of the intermediary 
matrix and the transform matrix not used in determining the intermediary 
matrix. 

[00137] In yet a further aspect, the present application provides a 
method of selecting transform matrices for use in a transcoder for down- 
sampling a DCT image directly in the DCT domain. The method comprises: a) 
obtaining a training set by applying a pre-selected down-sampling method to 
a DCT reference image to obtain a down-sampled DCT reference image; b) 
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initializing the transform matrices; and c) applying a learning with forgetting 
algorithm to the transform matrices to obtain transform matrices which 
reduce a learning objective function, wherein the learning objective function is 
a function of the visual quality of a down-sampled DCT image obtained using 
the transform matrices relative to the visual quality of the down-sampled DCT 
reference image, and the learning objective function is also a function of the 
computational complexity of applying the transform matrices. 

[00138] In yet another aspect, the present application provides an 
electronic device having a memory for storing transform matrices and a 
training set. The transform matrices include a row transform matrix and a 
column transform matrix. The training set includes at least one DCT 
reference Image and a corresponding down-sampled DCT reference image 
obtained using a spatial domain down-sampling method. The electronic 
device also includes a processor connected to the memory for obtaining 
transform matrices for down-sampling a DCT image directly in a DCT domain. 
The processor is configured to initialize the transform matrices and to apply a 
learning with forgetting algorithm to the transform matrices to obtain 
transform matrices which reduce a learning objective function. The learning 
objective function is a function of the visual quality of a down-sampled DCT 
image obtained using the transform matrices relative to the visual quality of 
the down-sampled DCT reference image. The learning objective function is 
also a function of the computational complexity associated with down- 
sampling using the transform matrices. 

[00139] In another aspect, the present application provides a method of 
selecting transform matrices for use In a transcoder for down-sampling a DCT 
image of size M x N to a down-sampled DCT image of size I x J directly in a 
DCT domain. The method comprises: a) selecting a spatial domain down- 
sampling method; b) applying the selected spatial domain down-sampling 
method to the DCT image to produce a down-sampled DCT reference image; 
and c) selecting transform matrices that minimize an optimization problem, 
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wherein the optimization problem is a function of the error between down- 
sampled Image obtained using the transform matrices and the down-sampled 
DCT reference image, and the optimization problem is also a function of the 
computational complexity of applying the transform matrices, the transform 
matrices comprising a row transform matrix and a column transform matrix. 

[00140] In another aspect, the present application provides an electronic 
device. The electronic device Includes a memory for storing transform 
matrices. The transform matrices include a row transform matrix and a 
column transform matrix. The electronic device further includes a processor 
connected to the memory for obtaining transform matrices for down-sampling 
a DCT Image of size M x N to a down-sampled DCT Image of size I x J directly 
In a DCT domain. The processor Is configured to select a spatial domain 
down-sampling method and to apply the selected spatial domain down- 
sampling method to the DCT image to produce a DCT reference Image. The 
processor is also configured to store transform matrices to the memory which 
minimize an optimization problem. The optimization problem is a function of 
the error between down-sampled image obtained using the transform 
matrices and the DCT reference image. The optimization problem Is also a 
function of the computational complexity of applying the transform matrices 
to down-sample the DCT image. 

[00141] Other aspects and features of the present application will be 
apparent to those of ordinary sl<iil in the art from a review of the following 
above description when considered in conjunction with the drawings. 

[00142] Certain adaptations and modifications of the invention will be 
obvious to those skilled in the art when considered in light of this description. 
Therefore, the above discussed embodiments are considered to be illustrative 
and not restrictive, the scope of the Invention being indicated by the 
appended claims rather than the foregoing description, and all changes which 
come within the meaning and range of equivalency of the claims are therefore 
intended to be embraced therein. 
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WHATIS CLAIMED IS: 

1. A method of selecting transform matrices for use in a transcader for 
down-sampling a DCT image of size M x N to a down-sampled DCT image of 
size I X J directly in a DCT domain, the method comprising: 

selecting a spatial domain down-sampling method; 

applying the selected spatial domain down-sampling method to the 
DCT image to produce a down-sampled DCT reference image; and 

selecting transform matrices that minimize an optimization problem, 
wherein the optimization problem is a function of the error between a down- 
sampled image obtained using the transform matrices and the down-sampled 
DCT reference image, and the optimization problem is also a function of the 
computational complexity of applying the transform matrices, the transform 
matrices comprising a row transform matrix and a column transform matrix. 

2. The method of claim 1, wherein the step of selecting includes 
determining the visual quality of down-sampled DCT images obtained by 
applying the row transform matrix and the column transform matrix to the 
DCT image, and determining the computational complexity associated with 
applying the row transform matrix and the column transform matrix to down- 
sample the DCT image. 

3. The method of claim 1, wherein the step of determining finds a solution 
to the optimization problem: 

where D^^^ is the row transform matrix; W^^j Is the column transform 
matrix; C^^ is the DCT image; D^^ ^^^y is the down-sampled DCT image 
obtained using the transform matrices; is the down-sampled DCT 
reference image obtained using another down-sampling method; is a 
measure of the computational complexity associated with using the transform 
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matrices Dfj^.W^^ to down-sample an image Cmn; and A is a quality and 

complexity trade-off parameter for balancing the trade-off between visual 
quality of the down-sampled DCT Image and the computational complexity of 
obtaining the down-sampled DCT Image. 

4. The method of claim 1 further comprising: 

retrieving a quality and complexity trade-off parameter, 

and wherein the step of determining includes using the quality and 
complexity trade-off parameter for balancing the visual quality of down- 
sampled DCT images with the computational complexity in obtaining down- 
sampled DCT images. 

5. The method of claim 1, wherein the computational complexity is a 
function of the absolute values of elements in the row-size transform matrix 
and the column transform matrix. 

6. The method of claim 5 further comprising a step of calculating the 
summation of the absolute value of all elements in the row transform matrix 
and the column transform matrix. 

7. The method of claim 1, wherein the computational complexity is a 
function of the absolute values of all elements In the row transform matrix 
and the column transform matrix that are below a selective learning with 
forgetting threshold parameter. 

8. The method of claim 7 further comprising a step of calculating the 
summation of the absolute values of all elements in the row transform matrix 
and the column transform matrix that are below the selective learning with 
forgetting threshold parameter. 

9. The method of claim 1, wherein the computational complexity is a 
function of the number of non-zero elements in a binary representation of 
each element of the row transform matrix and the column transform matrix. 

10. The method of claim 1, wherein the DCT image is a JPEG image. 
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11. The method of claim 1, wherein the DCT image is a frame of a video. 

12. The method of claim 11, -wherein the video Is of the DV video type. 

13. An electronic device comprising: 

a memory for storing transform matrices, the transform matrices 
comprising a row transform matrix and a column transform matrix; and 

a processor connected to the memory for obtaining transform matrices 
for down-sampling a DCT image of size M x N to a down-sampled DCT Image 
of size I X J directly In a DCT domain, the processor being configured to select 
a spatial domain down-sampling method and to apply the selected spatial 
domain down-sampling method to the DCT image to produce a DCT reference 
image, and to store transform matrices to the memory which minimize an 
optimization problem, the optimization problem being a function of the error 
between down-sampled image obtained using the transform matrices and the 
DCT reference image, the optimization problem also being a function of the 
computational complexity of applying the transform matrices to down-sample 
the DCT image. 

14. The electronic device of claim 13 wherein the optimization problem is: 

min (||D,^ 'W^ -V^f^X-r^), 

where is the row transform matrix; W^j Is the column transform 

matrix; C^^ is the DCT image; Dj^ C^ 'W^ Is the down-sampled DCT image 

obtained using the transform matrices; is the down-sampled DCT 

reference. image obtained using another down-sampling method; is a 

measure of the computational complexity associated with using the transform 
matrices D^^.W^j to down-sample an Image CM/v/and A is a quality and 

complexity trade-off parameter for balancing the trade-off between visual 
quality of the down-sampled DCT image and the computational complexity of 
obtaining the down-sampled DCT image. 
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15. The electronic device of claim 13, wherein the processor is further 
configured to determine the visual quality of down-sampled DCT Images 
obtained by applying the transform matrices to the DCT Image, and to 
determine the computational complexity associated with down-sampling the 
DCT image using the transform matrices. 

16. The electronic device of claim 13, wherein the processor is configured 
to determine transform matrices when there Is low processor usage. 

17. The electronic device of claim 13, wherein the electronic device Is a 
server which may be accessed by one or more wireless devices. 

18. The electronic device of claim 13, wherein the memory further stores a 
quality and complexity trade-off parameter, and wherein quality and 
complexity trade-off parameter is used In the optimization problem to balance 
the relative importance of quality and computational complexity. 

19. The electronic device of claim 13, wherein the computational 
complexity is a function of the absolute value of all elements in the row 
transform matrix and the column transform matrix, 

20. The electronic device of claim 13, wherein the processor is configured 
to ignore the complexity due to values In the transform matrices that are 
above a predetermined threshold. 
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AMENDED CLAIMS 
received by the International Bureau on 11 November 2008 (11.11.2008) 

1. A method of selecting transform matrices for use in a transcoder for 
down-sampling a DCT image of size M x N to a down-sampled DCT image of 
size I X J directly in a DCT domain, tlie method comprising: 

selecting a spatial domain down-sampling method; 

applying the selected spatial domain down-sampling method to the 
DCrr Image to produce a down-sampled DCT reference Image; and 

selecting transform matrices that minimize an optimization problem, 
wherein the optimization problem Is a function of the error between a down- 
sampled image obtained using the transform matrices and the down-sampled 
DCT reference image, and the optimization problem is also a function of the 
computational complexity of applying the transform matrices, the transform 
matrices comprising a row transform matrix and a column transform matrix. 

2. The method of claim 1, wherein the step of selecting includes 
determining the visual quality of down-sampled DCT images obtained by 
applying the row transform matrix and the column transform matrix to the 
DCT image, and determining the computational complexity associated with 
applying the row transform matrix and the column transform matrix to down- 
sample the DCT image, 

3. The method of claim i, wherein the step of selecting finds a solution to 
the optimization problem: 

where D^^^ Is the row transform matrix; Is the column transform 
matrix; C^^^ is the DCT Image; Dj^^-C^t^-W^ Is the down-sampled DCT Image 
obtained using the transform matrices; is the down-sampled DCT 
reference Image obtained using another down-sampling method; is a 
measure of the computational complexity associated with using the transform 
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matrices D^^^.W^ to down-sample an image C^; and ;i is a quality and 
complexity trade-off parameter for balancing the trade-off between visual 
quality of the down-sampled DCT image and the computational complexity of 
obtaining the down-sampled DCT image. 

4. The method of claim 1 further comprising: 

retrieving a quality and complexity trade-off parameter, 

and wherein the step of selecting includes using the quality and 
complexity trade-off parameter for balancing the visual quality of down- 
sampled DCT Images with the computational complexity in obtaining down- 
sampled DCT images. 

5. The method of claim 1, wherein the computational complexity Is a 
function of the absolute values of elements in the row transform matrix and 
the column transform matrix. 

6. The. method of claim 5 further comprising a step of calculating the 
summation of the absolute value of all elements in the row transform matrix 
and the column transform matrix. 

7. The method of claim 1, wherein the computational complexity Is a 
function of the absolute values of all elements In the row transform matrix 
and the column transform matrix that are below a selective learning with 
forgetting threshold parameter. 

8. The method of claim 7 further comprising a step of calculating the 
summation of the absolute values of all elements in the row transform matrix 
and the column transform matrix that are below the selective learning with 
forgetting threshold parameter. 

9. The method of claim 1, wherein the computational complexity is a 
function of the number of non-zero elements in a binary representation of 
each element of the row transform matrix and the column transform matrix. 

10. The method of claim i, wherein the DCT image is a JPEG image. 
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11. The method of claim 1, wherein the DCT image is a frame of a video. 

12. The method of claim 11, wherein the video is of the DV video type. 

13. An eiectronic device comprising; 

a memory for storing transform matrices, the transform matrices 
comprising a row transform matrix and a column transform matrix; and 

a processor connected to the memory for obtaining transform matrices 
for down-sampling a DCT image of size M x N to a down-sampled DCT image 
of size I X J directly in a DCT domain, the processor being configured to select 
a spatial domain down-sampling method and to apply the selected spatial 
domain down-sampling method to the DCT image to produce a DCT reference 
image, and to store transform matrices to the memory which minimize an 
optimization problem, the optimization problem being a function of the error 
between down-sampled image obtained using the transform matrices and the 
DCT reference image, the optimization problem also being a function of the 
computational complexity of applying the transform matrices to down-sample 
the DCT Image. 

14. The eiectronic device of claim 13 wherein the optimization problem is: 

where i)^^ is the row transform matrix; W^^j is the column transform 
matrix; C^^ is the DCT Image; D,^ -C^,^ is the down-sampled DCT Image 
obtained using the transform matrices; Vjj Is the down-sampled DCT 
reference image obtained using another down-sampling method; is a 
measure of the computational complexity associated with using the transform 
rhatrices to down-sample an image Cw/v/and A is a quality and 

complexity trade-off parameter for balancing the trade-off between visual 
quality of the down-sampled DCT image and the computational complexity of 
obtaining the down-sampled DCT Image. 



AMENDED SHEET (ARTICLE 19) 



wo 2008/148207 



43 



PCT/CA2008/001082 



15. The electronic device of claim 13, wherein the processor is further 
configured to determine the visual quality of down-sannpled DCT Images 
obtained by applying the transform matrices to the DCT image, and to 
determine the computational complexity associated with down-sampling the 
DCT image using the transform matrices. 

16. The electronic device of claim 13, wherein the processor is configured 
to determine transform matrices when there is low processor usage. 

17. The electronic device of claim 13, wherein the electronic device Is a 
server which may be accessed by one or more wireless devices. 

18. The electronic device of claim 13, wherein the memory further stores a 
quality and complexity trade-off parameter, and wherein quality and 
complexity trade-off parameter is used in the optimization problem to balance 
the relative importance of quality and computational complexity. 

19. The electronic device of claim 13, wherein the computational 
complexity is a function of the absolute value of all elements In the row 
transform matrix and the column transform matrix. 

20. The electronic device of claim 13, wherein the processor is configured 
to ignore the complexity due to values in the transform matrices that are 
above a predetermined threshold- 
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